Skip to content

Add Set-Like Operations to ElementArray#195

Merged
JeanLucPons merged 6 commits intomainfrom
194-add-set-like-operations-to-elementarray
Feb 18, 2026
Merged

Add Set-Like Operations to ElementArray#195
JeanLucPons merged 6 commits intomainfrom
194-add-set-like-operations-to-elementarray

Conversation

@gupichon
Copy link
Copy Markdown
Contributor

Description

This PR introduces native set-like operations on ElementArray, including support for boolean masks and automatic result typing.

The goal is to make element selection more expressive, algebraic, and robust, while preserving type specialization (e.g., MagnetArray, BPMArray) even when filtering heterogeneous arrays.

The following operators are now supported:

  • & → intersection between arrays
  • | → union (unique, stable order)
  • + → alias for union
  • - → difference between arrays
  • & mask → boolean filtering (keep True)
  • - mask → boolean removal (remove True)

Additionally, empty results now return a properly typed empty array instead of a Python list, ensuring safe chaining.


Related Issue

Features described there are:

  • new feature: set-like operations implemented via operator overloading in ElementArray
  • new feature: boolean mask support for & and -
  • new feature: automatic array typing using nearest common ancestor (MRO-based resolution)
  • improvement: empty results now return typed empty arrays instead of []

Changes to existing functionality

  • ElementArray.__auto_array reimplemented using nearest common ancestor resolution (MRO intersection) to correctly handle heterogeneous subclasses (e.g., multiple Magnet subclasses now correctly produce MagnetArray instead of ElementArray).
  • ElementArray.__create_array updated to return typed empty arrays instead of raw [] to preserve method chaining.
  • ElementArray.__init__ updated to always initialize internal attributes (__peer, __use_aggregator) even when the array is empty.
  • Added mask_by_type() to generate reusable boolean masks.
  • Added filter_by_type(), of_type(), and exclude_type() as declarative filtering helpers.
  • Ruff B905 compliance: all zip() calls updated to use strict=True.

These changes ensure:

  • Correct auto-typing even when multiple subclasses share a common base class.
  • Robust chaining even when intermediate results are empty.
  • Safe mask handling with explicit length validation.
  • No silent truncation due to zip() behavior.

Testing

The following pytest-compatible tests were added:

  • Intersection between two arrays returns correctly typed array
  • Union between arrays preserves uniqueness and order
  • Difference between arrays behaves correctly
  • & with Python boolean mask filters correctly
  • & with NumPy boolean mask filters correctly
  • - with boolean mask removes elements correctly
  • Mask length mismatch raises ValueError
  • Mixed subclasses of Magnet return MagnetArray (nearest common ancestor logic)
  • Empty result still returns typed array (chainable behavior)

All tests pass locally.


Verify that your checklist complies with the project

  • New and existing unit tests pass locally
  • Tests were added to prove that all features/changes are effective
  • The code is commented where appropriate
  • Any existing features are not broken (unless explicitly stated above)

@gupichon gupichon linked an issue Feb 16, 2026 that may be closed by this pull request
@gupichon gupichon self-assigned this Feb 16, 2026
@gupichon
Copy link
Copy Markdown
Contributor Author

@GamelinAl, @gubaidulinvadim I’ll let you take a look at the tests and check whether they suit your needs.

@gubaidulinvadim
Copy link
Copy Markdown
Contributor

gubaidulinvadim commented Feb 16, 2026

@gupichon It works really well. I've made a quick test with examples/SOLEIL_examples config file where we have different cells defines.

In [10]: all_bpms & cell1
Out[10]:
[BPM(peer='Simulator:design', name='BPM_001', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-SD/DG-EPOS/BPM.02/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-SD/DG-EPOS/BPM.02/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_002', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.03/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.03/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_003', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.04/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.04/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_004', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.05/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.05/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_005', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.06/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.06/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_006', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.07/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.07/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_007', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.08/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.08/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_008', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.09/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.09/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_009', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.10/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-AR/DG-EPOS/BPM.10/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_180', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-SD/DG-EPOS/BPM.01/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-SD/DG-EPOS/BPM.01/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None))]

In [11]: all_bpms & cell2
Out[11]:
[BPM(peer='Simulator:design', name='BPM_010', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN02-SD/DG-EPOS/BPM.01/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN02-SD/DG-EPOS/BPM.01/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_011', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN02-SD/DG-EPOS/BPM.02/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN02-SD/DG-EPOS/BPM.02/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_012', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.03/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.03/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_013', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.04/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.04/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_014', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.05/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.05/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_015', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.06/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.06/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 BPM(peer='Simulator:design', name='BPM_016', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.07/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN02-AR/DG-EPOS/BPM.07/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None))]

In [12]: cell1 & cell2
Out[12]: []

In [13]: cell1 | cell2
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[13], line 1
----> 1 cell1 | cell2

TypeError: unsupported operand type(s) for |: 'ElementArray' and 'ElementArray'

In [14]: cell1 + cell2
Out[14]:
[BPM(peer='Simulator:design', name='BPM_001', description=None, lattice_names=None, model=BPMSimpleModel(x_pos=AttributeReadOnly(attribute='AN01-SD/DG-EPOS/BPM.02/x_pos', unit='mm', range=None), y_pos=AttributeReadOnly(attribute='AN01-SD/DG-EPOS/BPM.02/y_pos', unit='mm', range=None), x_pos_index=None, y_pos_index=None)),
 CombinedFunctionMagnet(peer='Simulator:design', name='SH1_COR_001', description=None, lattice_names=None, mapping=[['B2', 'SH1_COR_001.sextupole'], ['B0', 'SH1_COR_001.hcorrector'], ['A0', 'SH1_COR_001.vcorrector']], model=IdentityCFMagnetModel(multipoles=['B2', 'B0', 'A0'], powerconverters=None, physics=[Attribute(attribute='AN01-AR/EM-SX/SHF.01/strength', unit='1/m**2', range=None), Attribute(attribute='AN01-AR/EM-COR/SHF.01-CDLH.01/strength', unit='1', range=None), Attribute(attribute='AN01-AR/EM-COR/SHF.01-CDLV.01/strength', unit='1', range=None)], units=['1/m**2', '1', '1'])),

For some reason, for Union operation I could not do '|' but the '+' has worked. I've cut the beginning IPython cells where I defined the array and the output of the last IPython cell to have a readable example.

Briefly looking at your tests, I don't see tests for | operation.

JeanLucPons
JeanLucPons previously approved these changes Feb 17, 2026
@JeanLucPons
Copy link
Copy Markdown
Contributor

For me it looks good.
However next time please separate renaming commit and code.
It would be nice to add some code snippet in the docstring.
Thanks

@gupichon
Copy link
Copy Markdown
Contributor Author

Yep, I completely forgot the | operator…
It’s now corrected, and the code snippets in the docstrings have been updated accordingly.

Copy link
Copy Markdown
Contributor

@gubaidulinvadim gubaidulinvadim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok to me. I'd let @GamelinAl to do his review because I'm not at all a user with a lot of strong demands.

@JeanLucPons
Copy link
Copy Markdown
Contributor

Could you use code block in your code snippet to get nice code in the doc ?
Thanks

    .. code-block:: python

        >>> hcorr = sr.live.get_magnets("HCORR")
        >>> vcorr = sr.live.get_magnets("VCORR")
        >>> all_corr = hcorr | vcorr

@gupichon
Copy link
Copy Markdown
Contributor Author

Could you use code block in your code snippet to get nice code in the doc ? Thanks

    .. code-block:: python

        >>> hcorr = sr.live.get_magnets("HCORR")
        >>> vcorr = sr.live.get_magnets("VCORR")
        >>> all_corr = hcorr | vcorr

Done

@JeanLucPons
Copy link
Copy Markdown
Contributor

It seems that there is a little issue, block should be indented.

image

@JeanLucPons
Copy link
Copy Markdown
Contributor

The doc is still failing. I do not manage to include :special_members: to document overloaded operator.

@gupichon
Copy link
Copy Markdown
Contributor Author

It’s the first time I’m working with Sphinx. It will be corrected in the next commit; I’ve managed to test it on my side.

@JeanLucPons
Copy link
Copy Markdown
Contributor

It is ok.

image

@JeanLucPons
Copy link
Copy Markdown
Contributor

To test the doc:
Just go to docs and make html

@JeanLucPons
Copy link
Copy Markdown
Contributor

Ah yes, it is possible to link as well

:py:class:`.ElementArray` instead of ElementArray

@gupichon
Copy link
Copy Markdown
Contributor Author

Yes, I just found where the issue was… Thanks for the commit.
I'm just waiting for the review now.

@JeanLucPons JeanLucPons self-requested a review February 17, 2026 14:44
@JeanLucPons
Copy link
Copy Markdown
Contributor

I think I cannot review as i did some commit.

@gubaidulinvadim
Copy link
Copy Markdown
Contributor

@JeanLucPons Alexis will make a review, he still may have a few comments. After we can merge.

@GamelinAl
Copy link
Copy Markdown
Contributor

I have no comment directly linked to this PR, everything I tested seem to work fine.

I have a few comments partially linked to this PR, up to you if we adress this here or if I should do a new issue.

  1. It seems that all Simulator/ControlSystem methods get_all_XXX, which I think are based on the ElementHolder class, returns a list and not arrays.
    A small example based on SOLEIL II example notebook:
control_mode = sr.design

bpms = control_mode.get_all_bpms() 
type(bpms) # returns list

bpms = control_mode.get_bpms("BPM")
type(bpms) # returns pyaml.arrays.bpm_array.BPMArray

From what I see, this is the case for all these methods, which prevent using the nice feature of the arrays.

  1. The methods of the ElementHolder class have no docstrings, but they are the ones exposed to the user here.
control_mode.get_bpms?
Signature: control_mode.get_bpms(name: str) -> pyaml.arrays.bpm_array.BPMArray
Docstring: <no docstring>
File:      ~/Python_dev/pyaml/pyaml/pyaml/common/element_holder.py
Type:      method
  1. More of a comment/question: maybe this is me not knowing how to use pyAML but I was thinking that string-matching was implemented at the array level, is it? I was expecting the following code to work:
bpms = control_mode.get_element("BPM*")
---------------------------------------------------------------------------
PyAMLException                            Traceback (most recent call last)
Cell In[74], line 1
----> 1 bpms = control_mode.get_element("BPM*")

File [~/Python_dev/pyaml/pyaml/pyaml/common/element_holder.py:109](http://localhost:8889/home/gamelina/Python_dev/pyaml/pyaml/pyaml/common/element_holder.py#line=108), in ElementHolder.get_element(self, name)
    108 def get_element(self, name: str) -> Element:
--> 109     return self.__get("Element", name, self.__ALL)

File [~/Python_dev/pyaml/pyaml/pyaml/common/element_holder.py:95](http://localhost:8889/home/gamelina/Python_dev/pyaml/pyaml/pyaml/common/element_holder.py#line=94), in ElementHolder.__get(self, what, name, array)
     93 def __get(self, what, name, array) -> Element:
     94     if name not in array:
---> 95         raise PyAMLException(f"{what} {name} not defined")
     96     return array[name]

PyAMLException: Element BPM* not defined

@JeanLucPons
Copy link
Copy Markdown
Contributor

@GamelinAl

Yes get_all_* function returns list of Element and the wildcard selection works only on Arrays.
Arrays are named and are part of the holder in order to be accessible by name from tuning or measurement tools.

For the time being, you can create standalone arrays (no name, not added to a holder) like below

    ae = ElementArray("", sr.design.get_all_elements())
    bpms = ae["BPM*"]  # Returns a standalone BPM array (on design)

or

    allBpms = BPMArray("", sr.design.get_all_bpms())

We can propose convenience functions for standalone arrays:

  # Returns a standalone `BPMArray` without name and not added to the holder
  bpms = Arrays.get( sr.design, "BPM*") 
  bpms = Arrays.get( sr.design.get_all_bpms() )
  # or
  bpms = sr.design.get_arrays( "BPM*") 
  bpms = sr.design.get_all_bpm_array( )

We can also change the interface of get_all_* function to return arrays with a default name, "all_bpms" or "all_magnets" for instance, and an optional argument for aggregators usage.

Concerning documentation, it is progressing...

@JeanLucPons JeanLucPons merged commit 4027ea4 into main Feb 18, 2026
3 checks passed
@gupichon
Copy link
Copy Markdown
Contributor Author

@GamelinAl, @JeanLucPons, @gubaidulinvadim, I suggest we discuss these points in a separate discussion before opening an issue. I think the ACCML concept of YellowPages, in addition to PyAML arrays, can cover these requirements.

@gupichon gupichon deleted the 194-add-set-like-operations-to-elementarray branch February 18, 2026 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Set-Like Operations to ElementArray

5 participants